With the increasing ability of large language models (LLMs), in-context learning (ICL) has become a new paradigm for natural language processing (NLP), where LLMs make predictions only based on contexts augmented with a few training examples. It has been a new trend exploring ICL to evaluate and extrapolate the ability of LLMs. In this paper, we aim to survey and summarize the progress, challenges, and future work in ICL. We first present a formal definition of ICL and clarify its correlation to related studies. Then, we organize and discuss advanced techniques of ICL, including training strategies, prompting strategies, and so on. Finally, we present the challenges of ICL and provide potential directions for further research. We hope our work can encourage more research on uncovering how ICL works and improving ICL in future work.

Supervised Deep-Learning (DL)-based reconstruction algorithms have shown state-of-the-art results for highly-undersampled dynamic Magnetic Resonance Imaging (MRI) reconstruction. However, the requirement of excessive high-quality ground-truth data hinders their applications due to the generalization problem. Recently, Implicit Neural Representation (INR) has appeared as a powerful DL-based tool for solving the inverse problem by characterizing the attributes of a signal as a continuous function of corresponding coordinates in an unsupervised manner. In this work, we proposed an INR-based method to improve dynamic MRI reconstruction from highly undersampled k-space data, which only takes spatiotemporal coordinates as inputs. Specifically, the proposed INR represents the dynamic MRI images as an implicit function and encodes them into neural networks. The weights of the network are learned from sparsely-acquired (k, t)-space data itself only, without external training datasets or prior images. Benefiting from the strong implicit continuity regularization of INR together with explicit regularization for low-rankness and sparsity, our proposed method outperforms the compared scan-specific methods at various acceleration factors. E.g., experiments on retrospective cardiac cine datasets show an improvement of 5.5 ~ 7.1 dB in PSNR for extremely high accelerations (up to 41.6-fold). The high-quality and inner continuity of the images provided by INR has great potential to further improve the spatiotemporal resolution of dynamic MRI, without the need of any training data.

Despite the surprising few-shot performance of in-context learning (ICL), it is still a common practice to randomly sample examples to serve as context. This paper advocates a new principle for ICL: self-adaptive in-context learning. The self-adaption mechanism is introduced to help each sample find an in-context example permutation (i.e., selection and ordering) that can derive the correct prediction, thus maximizing performance. To validate the effectiveness of self-adaptive ICL, we propose a general select-then-rank framework and instantiate it with new selection and ranking algorithms. Upon extensive evaluation on eight different NLP datasets, our self-adaptive ICL method achieves a 40% relative improvement over the common practice setting. Further analysis reveals the enormous potential of self-adaptive ICL that it might be able to close the gap between ICL and finetuning given more advanced algorithms. Our code is released to facilitate future research in this area: https://github.com/Shark-NLP/self-adaptive-ICL

Explaining the black-box predictions of NLP models naturally and accurately is an important open problem in natural language generation. These free-text explanations are expected to contain sufficient and carefully-selected evidence to form supportive arguments for predictions. Due to the superior generative capacity of large pretrained language models, recent work built on prompt engineering enables explanation generation without specific training. However, explanation generated through single-pass prompting often lacks sufficiency and conciseness. To address this problem, we develop an information bottleneck method EIB to produce refined explanations that are sufficient and concise. Our approach regenerates the free-text explanation by polishing the single-pass output from the pretrained language model but retaining the information that supports the contents being explained. Experiments on two out-of-domain tasks verify the effectiveness of EIB through automatic evaluation and thoroughly-conducted human evaluation.

用于提取和抽象性摘要系统的传统培训范例始终仅使用令牌级别或句子级培训目标。但是，始终从摘要级别评估输出摘要，从而导致培训和评估的不一致。在本文中，我们提出了一个基于对比度学习的重新排列框架，用于一阶段的摘要，称为COLO。通过建模对比目标，我们表明摘要模型能够根据摘要级别的分数直接生成摘要，而无需其他模块和参数。广泛的实验表明，CORO在CNN/DailyMail基准测试中提高了单阶段系统的提取和抽象结果，将其提高到44.58和46.33 Rouge-1得分，同时保留了参数效率和推断效率。与最先进的多阶段系统相比，我们节省了100多个GPU训练时间，并在推理期间获得3〜8加速比，同时保持可比的结果。

本文介绍了对体现药物（Genea）挑战2022的非语言行为的生成和评估的重生条目。Genea挑战提供了处理后的数据集并进行众包评估，以比较不同手势生成系统的性能。在本文中，我们探讨了基于多模式表示学习的自动手势生成系统。我们将WAVLM功能用于音频，FastText功能，用于文本，位置和旋转矩阵功能用于手势。每个模态都投影到两个不同的子空间：模态不变和特定于模态。为了学习模式间不变的共同点并捕获特定于模态表示的字符，在训练过程中使用了基于梯度逆转层的对抗分类器和模态重建解码器。手势解码器使用与音频中的节奏相关的所有表示和功能生成适当的手势。我们的代码，预培训的模型和演示可在https://github.com/youngseng/represture上找到。

只有单个目标扬声器的语音供参考的单发语音转换（VC）已成为一个热门研究主题。现有作品通常会散布音色，而有关音高，节奏和内容的信息仍然混合在一起。为了进一步删除这些语音组件，有效地执行一声VC，我们采用随机重新采样用于音高和内容编码器，并使用互信息的各种对比对数比率上限和基于梯度反向层的对抗性相互信息学习来确保不同部分在训练过程中仅包含所需的分离表示的潜在空间。 VCTK数据集的实验显示该模型就自然性和智能性方面实现了一声VC的最新性能。此外，我们可以通过语音表示分离分别传递音色，音调和节奏的单发VC的特征。我们的代码，预训练的模型和演示可在https://im1eon.github.io/is2022-Srdvc/上获得。

跨言扬声器风格的转移旨在提取给定参考语音的语音样式，可以在任意目标扬声器的音色中复制。有关此主题的现有方法已经探索了利用语音级样式标签通过全球或本地规模样式表示进行样式转移。但是，有声读物数据集通常以本地韵律和全球类型的形式进行特征，并且很少伴有发言级风格的标签。因此，正确地将阅读方式转移到不同的扬声器上仍然是一项具有挑战性的任务。本文旨在介绍块的多尺度跨言式风格模型，以捕获有声读物的全球类型和本地韵律。此外，通过使用拟议的可切换对手分类器来解开扬声器的音色和样式，提取的阅读样式可适应不同扬声器的音色。实验结果证实，该模型设法将给定的阅读方式转移到新的目标扬声器上。在局部韵律和全球流派类型预测指标的支持下，进一步揭示了所提出的方法在多扬声器有声读物中的潜力。

已经提出了使用锚定参考样品（ORAR）的序数回归，以自动预测输入刺激的主观平均意见评分（MOS）。 Orars通过将测试样本与每个预定的锚定参考样本配对来解决MOS预测问题。然后，使用训练有素的二进制分类器来预测哪种样品，测试或锚定在统计上更好。然后，使用二进制偏好决定的后者来预测测试样品的MOS。在本文中，提出了严格的框架，分析和实验，以证明Orars在简单的回归中具有优势。这项工作的贡献是：1）表明可以将传统的回归重新构成多个偏好测试以产生更好的性能，这可以通过实验中的模拟来确认； 2）将Orars概括为其他回归问题并验证其有效性； 3）提供一些可以确保正确应用Orars的条件。

近年来见证了自动扬声器验证（ASV）的非凡发展。但是，先前的作品表明，最新的ASV模型非常容易受到语音欺骗的攻击，而最近提出的高性能欺骗对策（CM）模型仅专注于独立的反欺骗任务，而忽略了该模型随后的发言人验证过程。如何将CM和ASV集成在一起仍然是一个悬而未决的问题。最近发生了欺骗意识的说话者验证（SASV）挑战，即当共同优化CM和ASV子系统时，可以提供更好的性能。在挑战的情况下，参与者提出的集成系统必须同时拒绝冒名顶替者和欺骗目标扬声器的攻击，这些攻击者直觉有效地与可靠，欺骗的ASV系统的期望相匹配。这项工作着重于基于融合的SASV解决方案，并提出了一个多模型融合框架，以利用多个最先进的ASV和CM模型的功能。拟议的框架将SASV-EER从8.75％提高到1.17 \％，与SASV挑战中最佳基线系统相比，相对改善为86％。